Methods of and units for motion or depth estimation and image processing apparatus provided with such motion estimation unit

ABSTRACT

In the method of motion estimation a tree ( 120 ) of segments ( 102 - 114 ) of an image ( 100 ) is generated by performing a hierarchical segmentation. The tree ( 120 ) of segments is analyzed to control the generation of a set ( 118 ) of candidate motion vectors of a segment ( 104 ). For the motion vectors of the set ( 118 ) match penalties are calculated. Finally, a particular motion vector ( 116 ) is selected from the set ( 118 ) of candidate motion vectors on the basis of match penalties. In the method of depth estimation depth data is calculated on the basis of a motion vector and the rules of parallax.

[0001] The invention relates to a method of motion estimation ofsegments of an image, comprising the steps of:

[0002] generating a set of candidate motion vectors of a particularsegment;

[0003] computing for each candidate motion vector a match penalty; and

[0004] selecting a particular motion vector from the set of candidatemotion vectors on the basis of match penalties.

[0005] The invention further relates to a motion estimator unit formotion estimation of segments of an image, comprising:

[0006] a generator for generating a set of candidate motion vectors of aparticular segment;

[0007] a computing means for computing for each candidate motion vectora match penalty; and

[0008] a selecting means for selecting a particular motion vector fromthe set of candidate motion vectors on the basis of match penalties.

[0009] The invention further relates to a method of depth estimation ofsegments of an image, comprising the steps of:

[0010] generating a set of candidate motion vectors of a particularsegment;

[0011] computing for each candidate motion vector a match penalty;

[0012] selecting a particular motion vector from the set of candidatemotion vectors on the basis of match penalties; and

[0013] calculating depth data of the particular segment on the basis ofthe particular motion vector.

[0014] The invention further relates to a depth estimator unit for depthestimation of segments of an image, comprising:

[0015] a generator for generating a set of candidate motion vectors of aparticular segment;

[0016] a computing means for computing for each candidate motion vectora match penalty;

[0017] a selecting means for selecting a particular motion vector fromthe set of candidate motion vectors on the basis of match penalties; and

[0018] a depth calculating means for calculating depth data of theparticular segment on the basis of the particular motion vector.

[0019] The invention further relates to an image processing apparatuscomprising:

[0020] a motion estimator unit for motion estimation of segments of animage, comprising:

[0021] a generator for generating a set of candidate motion vectors ofthe particular segment;

[0022] a computing means for computing for each candidate motion vectora match penalty; and

[0023] a selecting means for selecting a particular motion vector fromthe set of candidate motion vectors on the basis of match penalties; and

[0024] a motion compensated image processing unit.

[0025] A method of motion estimation of the kind described in theopening paragraph is known from the article “True-Motion Estimation with3-D Recursive Search Block Matching” by G. de Haan et. al. in IEEETransactions on circuits and systems for video technology, vol.3, no.5,October 1993, pages 368-379.

[0026] Motion estimation has been used with success in videoapplications for motion compensated image processing like scan-rateup-conversion, de-interlacing and temporal noise reduction. A categoryof motion estimation algorithms is called “Block matching”. Blockmatching algorithms are iterative minimization algorithms which assumethat all pixels within a given block move uniformly. For that block amatch penalty is minimized with respect to possible motion vectorcandidates. A match penalty might be e.g. the Sum of Absolutedifferences (SAD) or Mean Squared Error (MSE). Typically, blocks are 8by 8 pixels. However the principle of estimating motion vectors byminimization of a match penalty works also for segments with anirregular shape. In that case, the motion estimation/match penaltycomputation is segment based. Segment boundaries might be aligned withluminosity or color discontinuities. In this way, segments can beinterpreted as being objects or parts of objects in the scene.

[0027] The speed of the method of finding the best motion vectors of thevarious segments of an images depends on the number of segments, thesize of these segments and on the number of possible candidate motionvectors of each segment. The accuracy depends on two factors:

[0028] Firstly, the size of the segments, since larger segments are lessprone to noise. However the size is restricted by the fact that themotion of the segment should be constant. Hence, segments shouldpreferably coincide with the largest region still having the samemotion.

[0029] Secondly, the choice of the candidate motion vectors. If only alimited number of candidate motion vectors is tested, a good choice ofthe candidate motion vectors is crucial. In the cited article thecandidate set comprises the current motion vectors of neighboringblocks, augmented with some random candidate motion vectors. Theunderlying assumption is that neighboring blocks may have a similarmotion. However, neighboring blocks are only likely to have the samemotion if they belong to the same object. Hence, trying candidate motionvectors which are not likely to belong to the same object is wastedcomputational effort. It is a disadvantage of the known method that timeand computational effort are wasted.

[0030] It is known that the steps of a method of motion estimation maybe followed by a calculating step to obtain a method of depthestimation. The following problem is considered: given a series ofimages of a static scene taken by a camera with known motion, depthinformation should be recovered. All apparent motion in the series ofimages results from parallax. Differences in motion between one segmentand another indicate a depth difference. Indeed, analyzing twoconsecutive images, the parallax between a given image segment at time tand the same segment at t+1 can be computed. This parallax correspondsto the motion of different parts of the scene. In the case oftranslation of the camera, objects in the foreground move more thanthose in the background. By applying geometrical relations, the depthinformation can be deduced from the motion. This concept is described byP. Wilinski and K. van Overveld in the article “Depth from motion usingconfidence based block matching” in Proceedings of Image andMultidimensional Signal Processing Workshop, pages 159-162, Alpbach,Austria, 1998.

[0031] It is a first object of the invention to provide a method ofmotion estimation of the kind described in the opening paragraph with animproved performance.

[0032] It is a second object of the invention to provide a motionestimator unit of the kind described in the opening paragraph with animproved performance.

[0033] It is a third object of the invention to provide a method ofdepth estimation of the kind described in the opening paragraph with animproved performance.

[0034] It is a fourth object of the invention to provide a depthestimator unit of the kind described in the opening paragraph with animproved performance.

[0035] It is a fifth object of the invention to provide an imageprocessing apparatus of the kind described in the opening paragraph withan improved performance.

[0036] The first object of the invention is achieved in that the methodof motion estimation further comprises the steps of:

[0037] generating a tree of the segments of the image by performing ahierarchical segmentation; and

[0038] analyzing the tree of the segments to control the generation ofthe set of candidate motion vectors of the particular segment. Thehierarchical segmentation provides information on the likeliness ofsegments belonging to the same object due to the tree structure of thesegmentation. The tree has a single node at the highest level, which iscalled the root. Each node in the tree has a unique parent at a nexthigher level. Each segment corresponds with a node of the tree. A parentsegment has child segments. An example of a hierarchical segmentationmethod is known from U.S. Pat. No. 5,867,605. The performance, i.e.speed and/or accuracy of the method of motion estimation can be improvedwith the aid of the tree in two manners: top-down and bottom-up. Theadvantage in both cases is that the number of candidate vectors forwhich match penalties have to be calculated is relatively low.

[0039] In an embodiment of the method of motion estimation according tothe invention, the motion vectors of the segments of the tree areestimated by processing the tree of the segments recursively in atop-down direction. This may be done according to the followingprocedure:

[0040] The estimation of the appropriate motion vectors starts on highlevel of the tree, where the segments are still large.

[0041] If no satisfactory motion vector is found for a segment, thenmotion vectors are estimated for all children of this node separately.

[0042] The previous step is repeated recursively until satisfactorymotion vectors are obtained.

[0043] In this manner, the number of segments to be matched is minimal,without sacrificing the accuracy. The process of estimating motionvectors continues until satisfactory motion vectors are obtained. Theresult is that only motion vectors are estimated for segments at arelatively high level in the tree. The advantage is that the processstops with the largest, and thus least noise-prone, segments possible.

[0044] In a modification of the embodiment of the method of motionestimation according to the invention, the motion vectors are estimatedof segments, corresponding to children of a node of the tree thatcorresponds to the particular segment, if a match penalty of theparticular motion vector is unsatisfactory. The advantage of this stopcriterion is its simplicity. Another criterion might be based oncomparing match penalties of the motion vector of a parent segment andthe motion vectors of the children segments. However applying thislatter criterion implies extra computations.

[0045] In an embodiment of the method of motion estimation according tothe invention, the motion vectors of the segments of the tree areestimated by processing the tree of the segments in a bottom-updirection. A least common ancestor of two segments is defined as thelowest node in the tree which is on the path from root to each of thosesegments. Segments which have a least common ancestor at a low positionin the tree are more likely to belong to the same object than segmentswhich have a least common ancestor high in the tree. This informationcan be used to restrict the number of candidate motion vectors by takingonly the relevant neighbors into account.

[0046] In an embodiment of the method of motion estimation according tothe invention, the step of generating the set of candidate motionvectors of the particular segment comprises the following sub-steps of:

[0047] generating an initial list of candidate motion vectors comprisingmotion vectors of neighboring segments;

[0048] ordering the initial list of candidate motion vectors accordingto positions of the corresponding nodes within the tree; and

[0049] restricting the initial list of candidate motion vectors tocandidate motion vectors with a relatively high order, resulting in theset of candidate motion vectors. The motion vectors of the neighboringsegments are put in order of importance. The importance is related tothe position in the tree of the least common ancestor of the particularsegment and the neighboring segment under consideration. The lower theposition of the least common ancestor in the tree, the more importantthe candidate motion vector The second object of the invention isachieved in that the motion estimator unit further comprises:

[0050] a segmentation means for generating a tree of the segments of theimage, designed to perform a hierarchical segmentation; and

[0051] an analyzing means for analyzing the tree of the segments tocontrol the generator.

[0052] Modifications of the motion estimator unit and variations thereofmay correspond to modifications and variations thereof of the methoddescribed.

[0053] The third object of the invention is achieved in that the methodof depth estimation further comprises the steps of:

[0054] generating a tree of the segments of the image by performing ahierarchical segmentation; and

[0055] analyzing the tree of the segments to control the generation ofthe set of candidate motion vectors of the particular segment.

[0056] The fourth object of the invention is achieved in that the depthestimator unit further comprises:

[0057] a segmentation means for generating a tree of the segments of theimage, designed to perform a hierarchical segmentation; and

[0058] an analyzing means for analyzing the tree of the segments tocontrol the generator.

[0059] The fifth object of the invention is achieved in that the motionestimator unit of the image processing apparatus further comprises:

[0060] a segmentation means for generating a tree of the segments of theimage, designed to perform a hierarchical segmentation; and

[0061] an analyzing means for analyzing the tree of the segments tocontrol the generator.

[0062] These and other aspects of the methods of and units for motion ordepth estimation and of the image processing apparatus according to theinvention will become apparent from and will be elucidated with respectto the implementations and embodiments described hereinafter and withreference to the accompanying drawings, wherein:

[0063]FIG. 1 shows an image segmented into 8 segments and thecorresponding tree;

[0064]FIG. 2A schematically shows the segments for which motion vectorsare estimated in the case of a top-down approach, at the second highestlevel of hierarchy of the tree;

[0065]FIG. 2B schematically shows the segments for which motion vectorsare estimated in the case of a top-down approach, at the third highestlevel of hierarchy of the tree;

[0066]FIG. 2C schematically shows the segments for which motion vectorsare estimated in the case of a top-down approach, at the lowest level ofhierarchy of the tree.

[0067]FIG. 3A schematically shows elements of a motion estimatordesigned for a top-down approach;

[0068]FIG. 3B schematically shows elements of a motion estimator unitdesigned for a bottom-up approach;

[0069]FIG. 4 schematically shows elements of a depth estimator unit; and

[0070]FIG. 5 schematically shows elements of an image processingapparatus. Corresponding reference numerals have the same meaning in allof the Figures.

[0071]FIG. 1 shows an image 100 hierarchically segmented into 8 segments102-114 and the corresponding tree 120. The tree has 13 nodes 122-146from which node 122 is called the root. Each node 122-146 corresponds toa segment of the image 100. For some nodes it is indicated to whichsegment of the image 100 it corresponds, e.g. node 130 corresponds tosegment 102. References are provided for convenience, A-H.

[0072] The concept of hierarchical segmentation will be explainedbriefly. Hierarchical segmentation is a region-based segmentationtechnique in which segments in the image are determined that satisfy acertain homogeneity criterion. Subsequently, these segments areextended, e.g. grown by continuously varying the homogeneity parameter,such that the union of segments covers the whole image. Then thesegmentation is complete. The basic approach consists of the followingsteps:

[0073] A criterion for the homogeneity is chosen, e.g. the variance ofthe luminosity in a segment around a pixel. This means that it isdecided whether or not a pixel can be considered as part of on initialsegment on the basis of the luminosity variance in a region around thepixel.

[0074] For each value of the homogeneity criterion the resultingsegments can be calculated. However preferably it is done for somepre-determined thresholds. Each threshold corresponds with a level ofhierarchy in the tree.

[0075] A tree structure is built for the segments. Note that if for acertain value of the threshold a segment exists, this segment existsalso for all higher values of the threshold. If the threshold increases,segments can grow larger, and segments which are separated for a lowvalue for the threshold might merge at a higher level of the threshold.However, because of the way the segments are created, it can neverhappen that segments decrease in size or are split if the thresholdincreases.

[0076] Optionally this tree is pruned to achieve separated segments.

[0077] Optionally segments are grown by means of a morphologicaloperation. The free parameter is the threshold of the homogeneitycriterion. This criterion can be specified in a very intuitive way, e.g.variance of the luminosity in a certain environment, similarity ofneighboring pixels, etc. If segments have to be merged, the treestructure specifies uniquely which segments are to be merged to arriveat larger segments. Finally, the tree itself gives a structural relationbetween the segments.

[0078] The advantage of the hierarchical segmentation, resulting in thetree 120, for the estimation of the motion vector 116 of segment C 104is described below. The method of motion estimation according to thebottom-up approach comprises the following steps:

[0079] For segment C 104 a set CS_(C) of candidate motion vectors V_(l)is generated, where V_(l) is the motion vector of a neighboring segmentof C 104, i.e. 102,103,106-114.

[0080] The candidate motion vectors are ordered according to theposition of the least common ancestor in the tree 120 of node C 140 andthe neighboring segment under consideration. E.g. node 132 is the leastcommon ancestor of node C 140 and node D 142. The lower the position ofthe least common ancestor in the tree, the more relevant the candidatemotion vectors V_(l).

[0081] The set CS_(C) of candidate motion vectors is restricted to onlythose K motion vectors V_(l) which have the highest relevance.

[0082] In the situation of the prior art, the set of candidate motionvectors of segment C 104 would be CS_(C)={V_(A), V_(B), V_(D), V_(E),V_(F), V_(G), V_(H), V_(R)} where V_(R) is a random motion vector.However according to the invention it is allowed that CS_(C)={V_(B),V_(D), V_(A), V_(R)} is taken, since from the tree structure it can bederived that the least common ancestor 122 of node C 140 and, e.g. nodeE 134 is so high up in the tree 120 that it is unlikely that segment C104 and segment E 108 belong to the same object and have the samemotion. In this case, K=3 is taken. In this manner, it is possible torestrict the number of candidate motion vectors to be tested. Since theselection of the candidate motion vectors is based on the tree structureof the segmentation, only the information from the most relevantneighboring segments is used. The result is a gain in efficiency, i.e.less candidate motion vectors to be tested without a loss in accuracy,since the non-tested candidate motion vectors are from segments whichprobably are part of different objects and thus less likely to have asimilar motion.

[0083]FIG. 2A schematically shows the segments 202,204 and 206 for whichmotion vectors 208,210 and 212 are estimated. The tree 120 in FIG. 2Ashows a state in the method according to a top-down approach: the secondhighest level of hierarchy of the tree. The nodes 124,126 and 128correspond with the segments for which motion vectors are estimated. Theprinciple of the top-down approach is described in connection with FIG.3A.

[0084]FIG. 2B schematically shows the segments 202,216,214,218 and 220for which motion vectors 208,222,224,226 and 228 are estimated. The tree120 in FIG. 2B shows a state in the method according to a top-downapproach: the third highest level of hierarchy of the tree. The nodes128,130,132,134 and 135 correspond with the segments for which motionvectors are estimated. By comparing FIGS. 2A and 2B the effect ofsplitting segments into smaller segments on motion vectors can be seen.E.g. a motion vector 210 was estimated for segment 204 and the motionvectors 226 and 228 are estimated for the segments 214 respectively 216.E.g. a motion vector 212 was estimated for segment 206 and the motionvectors 222 and 224 are estimated for the segments 220 respectively 218.

[0085]FIG. 2C schematically shows the segments 202,216,218,230,232,234and 236 for which motion vectors 208,224,228,238,240,244 and 242 areestimated. The tree 120 in FIG. 2C shows a state in the method accordingto a top-down approach: the lowest level of hierarchy of the tree. Thenodes 128,130,134,138,140,142,144 and 146 correspond with the segmentsfor which motion vectors are estimated. By comparing FIGS. 2B and 2C theeffect of splitting segments into smaller segments on motion vectors canbe seen. E.g. a motion vector 226 was estimated for the segment 214 andthe motion vectors 238 and 240 are estimated for the segments 230respectively 232. A motion vector 222 was estimated for the segment 220and the motion vectors 242 and 244 are estimated for the segments 236respectively 234.

[0086]FIG. 3A schematically shows elements of a motion estimator unit300 designed for a top-down approach. The motion estimator unit 300comprises:

[0087] a generator 310 for generating a set of candidate motion vectorsof a particular segment;

[0088] a computing means 306 for computing for each candidate motionvector a match penalty;

[0089] a selecting means 308 for selecting a particular motion vectorfrom the set of candidate motion vectors on the basis of matchpenalties;

[0090] a segmentation means 302 for generating a tree of the segments ofthe image, designed to perform a hierarchical segmentation; and

[0091] an analyzing means 304 for analyzing the tree of the segments tocontrol the generator 310.

[0092] The input of the motion estimator unit 300 comprises images andis provided at the input connector 312. The output of the motionestimator unit 300 are motion vectors of the segments. The behavior ofthe motion estimator unit 300 is as follows. First a hierarchicalsegmentation is performed by the segmentation means 302. Thissegmentation is performed as described in U.S. Pat. No. 5,867,605. Theresult of the segmentation is a tree of segments. The motion vectors ofthe segments of the tree are estimated by processing the tree of thesegments recursively in a top-down direction. This is done according tothe following procedure:

[0093] The estimation of the appropriate motion vectors starts on highlevel of the tree, where the segments are still large.

[0094] If no satisfactory motion vector is found for a segment, thenmotion vectors are estimated for all children of this node separately. Acriterion for a satisfactory motion vector is that the match penalty ofthe particular motion vector is less then a predetermined threshold.

[0095] The previous step is repeated recursively until satisfactorymotion vectors are obtained.

[0096] In this manner, the number of segments to be matched is minimal,without sacrificing the accuracy. The process of estimating motionvectors continues until satisfactory motion vectors are obtained. Theresult is that only motion vectors are estimated for segments at arelatively high level in the tree. The advantage is that the processstops with the largest, and thus least noise-prone, segments possible.

[0097]FIG. 3B schematically shows elements of a motion estimator unit301 designed for a bottom-up approach. Most of the elements of thismotion estimator unit 301 are equal to those of the motion estimatorunit 300 as described in FIG. 3A. The first generating means 310 forgenerating the set of candidate motion vectors comprises:

[0098] a second generating means 303 for generating an initial list ofcandidate motion vectors comprising motion vectors of neighboringsegments;

[0099] an ordering means 305 for ordering the initial list of candidatemotion vectors according to positions of the corresponding nodes withinthe tree; and

[0100] a filter 307 to restrict the initial list of candidate motionvectors to candidate motion vectors with a relatively high order.

[0101] This tree is processed in a bottom-up approach. This means thatfor the leaves of the tree, i.e. smallest segments, sets of candidatemotion vectors are determined as described in FIG. 1. When a set ofmotion vectors is determined for a particular segment the matchpenalties are calculated by the computing means 306. Then the selectingmeans 308 selects a particular motion vector from the set of candidatemotion vectors on the basis of match penalties.

[0102]FIG. 4 schematically shows elements of a depth estimator unit 400.The depth estimator unit 400 comprises:

[0103] a first generating means 310 for generating a set of candidatemotion vectors of a particular segment;

[0104] a computing means 306 for computing for each candidate motionvector a match penalty;

[0105] a selecting means 308 for selecting a particular motion vectorfrom the set of candidate motion vectors on the basis of matchpenalties;

[0106] a segmentation means 302 for generating a tree of the segments ofthe image, designed to perform a hierarchical segmentation;

[0107] an analyzing means 304 for analyzing the tree of the segments tocontrol the first generating means; and

[0108] a depth calculating means 402 for calculating depth data of theparticular segment on the basis of the particular motion vector. Thedepth data might comprise a scalar depth value or a scalar depth valueand an orientation of the object.

[0109] The input of the motion estimator unit 300 comprises images whichare provided at the input connector 312 and camera calibration datawhich is provided at the input connector 416. The output of the motionestimator unit 300 are depth values for the segments. The behavior ofthe depth estimator unit 300 is as follows. First a hierarchicalsegmentation is performed by the segmentation means 302. Thissegmentation is performed as described in U.S. Pat. No. 5,867,605. Theresult of the segmentation is a tree of the segments that correspondwith objects or parts of objects in the scene. For a particular objectit is assumed that it has a depth relative to a pre-determined origin,e.g. the camera, with a value in a range of candidate values. This depthis related to motion of objects in the images. That means that if themotion of the object relative to the camera is known, then the depth canbe estimated on the rules of parallax. This implies that motion vectorshave to be calculated for estimating depth. The method of depthestimation is according to the method described in the article “Depthfrom motion using confidence based block matching” in Proceedings ofImage and Multidimensional Signal Processing Workshop, pages 159-162,Alpbach, Austria, 1998. In FIG. 4 it is depicted that the selectingmeans 308 provides the particular motion vector of a segment to thedepth calculating means 402. The motion estimation can be as describedin FIG. 3A or as described in FIG. 3B.

[0110]FIG. 5 schematically shows elements of an image processingapparatus 500 comprising:

[0111] receiving means 501 for receiving a signal representing images tobe displayed after some processing has been performed. The signal may bea broadcast signal received via an antenna or cable but may also be asignal from a storage device like a VCR (Video Cassette Recorder) orDigital Versatile Disk (DVD). The signal is provided at the inputconnector 506.

[0112] a motion estimator unit 300 or 301 as described in connectionwith FIG. 3A respectively FIG. 3B;

[0113] a motion compensated image processing unit 502; and

[0114] a display device for displaying the processed images. The motioncompensated image processing unit 502 supports the following types ofprocessing:

[0115] De-interlacing: Interlacing is the common video broadcastprocedure for transmitting the odd or even numbered image linesalternately. De-interlacing attempts to restore the full verticalresolution, i.e. make odd and even lines available simultaneously foreach image;

[0116] Up-conversion: From a series of original input images a largerseries of output images is calculated. Output images are temporallylocated between two original input images; and

[0117] Temporal noise reduction.

[0118] The motion compensated image processing unit 502 requires imagesand motion vectors as its input. In case of a 3D display device, e.g.with a lenticular screen, the depth values can be used to render 3Dimages.

[0119] It should be noted that the above-mentioned embodimentsillustrate rather than limit the invention and that those skilled in theart will be able to design alternative embodiments without departingfrom the scope of the appended claims. In the claims, any referencesigns placed between parentheses shall not be constructed as limitingthe claim. The word ‘comprising’ does not exclude the presence ofelements or steps not listed in a claim. The word “a” or “an” precedingan element does not exclude the presence of a plurality of suchelements. The invention can be implemented by means of hardwarecomprising several distinct elements and by means of a suitablyprogrammed computer. In the unit claims enumerating several means,several of these means can be embodied by one and the same item ofhardware.

1. A method of motion estimation of segments (102-114) of an image(100), comprising the steps of: generating a set (118) of candidatemotion vectors of a particular segment (104); computing for eachcandidate motion vector a match penalty; and selecting a particularmotion vector (116) from the set (118) of candidate motion vectors onthe basis of match penalties, characterized in further comprising thesteps of: generating a tree (120) of the segments (102-114) of the image(100) by performing a hierarchical segmentation; and analyzing the tree(120) of the segments (102-114) to control the generation of the set(118) of candidate motion vectors of the particular segment (104).
 2. Amethod of motion estimation as claimed in claim 1, characterized in thatmotion vectors of the segments (102-114) of the tree (120) are estimatedby processing the tree (120) of the segments (102-114) recursively in atop-down direction.
 3. A method of motion estimation as claimed in claim2, characterized in that motion vectors are estimated of segments(102-114), corresponding to children of a node of the tree (120) thatcorresponds to the particular segment (104), if a match penalty of theparticular motion vector (116) is unsatisfactory.
 4. A method of motionestimation as claimed in claim 1, characterized in that the motionvectors of the segments (102-114) of the tree (120) are estimated byprocessing the tree (120) of the segments (102-114) in a bottom-updirection.
 5. A method of motion estimation as claimed in claim 4,characterized in that the step of generating the set (118) of candidatemotion vectors of the particular segment (104) comprises the followingsub-steps of: generating an initial list of candidate motion vectorscomprising motion vectors of neighboring segments (102-114); orderingthe initial list of candidate motion vectors according to positions ofthe corresponding nodes (122-146) within the tree (120); and restrictingthe initial list of candidate motion vectors to candidate motion vectorswith a relatively high order, resulting in the set (118) of candidatemotion vectors.
 6. A motion estimator unit (300,301) for motionestimation of segments (102-114) of an image, comprising: a firstgenerating means (310) for generating a set (118) of candidate motionvectors of a particular segment (104); a computing means (306) forcomputing for each candidate motion vector a match penalty; and aselecting means (308) for selecting a particular motion vector (116)from the set (118) of candidate motion vectors on the basis of matchpenalties, characterized in further comprising: a segmentation means(302) for generating a tree (120) of the segments (102-114) of theimage, designed to perform a hierarchical segmentation; and an analyzingmeans (304) for analyzing the tree (120) of the segments (102-114) tocontrol the first generating means (310).
 7. A motion estimator unit(300) as claimed in claim 6, characterized in being designed to estimatemotion vectors of the segments (102-114) of the tree (120) by processingthe tree (120) of the segments (102-114) recursively in a top-downdirection.
 8. A motion estimator unit (300) as claimed in claim 7,characterized in being designed to estimate motion vectors of segments(102-114), corresponding to children of a node of the tree (120) thatcorresponds to the particular segment (104), if a match penalty of theparticular motion vector (116) is unsatisfactory.
 9. A motion estimatorunit (301) as claimed in claim 6, characterized in being designed toestimate motion vectors of the segments (102-114) of the tree (120) byprocessing the tree (120) of the segments (102-114) in a bottom-updirection.
 10. A motion estimator unit (301) as claimed in claim 9,characterized in that the first generating means (310) for generatingthe set (118) of candidate motion vectors of the particular segment(104) comprises: a second generating means (303) for generating aninitial list of candidate motion vectors comprising motion vectors ofneighboring segments (102-114); an ordering means (305) for ordering theinitial list of candidate motion vectors according to positions of thecorresponding nodes (122-146) within the tree (120); and a filter (307)to restrict the initial list of candidate motion vectors to candidatemotion vectors with a relatively high order.
 11. A method of depthestimation of segments (102-114) of an image, comprising the steps of:generating a set (118) of candidate motion vectors of a particularsegment (104); computing for each candidate motion vector a matchpenalty; selecting a particular motion vector (116) from the set (118)of candidate motion vectors on the basis of match penalties; andcalculating depth data of the particular segment (104) on the basis ofthe particular motion vector (116), characterized in further comprisingthe steps of: generating a tree (120) of the segments (102-114) of theimage (100) by performing a hierarchical segmentation; and analyzing thetree (120) of the segments (102-114) to control the generation of theset (118) of candidate motion vectors of the particular segment (104).12. A depth estimator unit (400) for depth estimation of segments(102-114) of an image, comprising: a first generating means (310) forgenerating a set (118) of candidate motion vectors of a particularsegment (104); a computing means (306) for computing for each candidatemotion vector a match penalty; a selecting means (308) for selecting aparticular motion vector (116) from the set (118) of candidate motionvectors on the basis of match penalties; and a depth calculating means(402) for calculating depth data of the particular segment (104) on thebasis of the particular motion vector (116), characterized in furthercomprising: a segmentation means (302) for generating a tree (120) ofthe segments (102-114) for the image (100) by performing a hierarchicalsegmentation; and an analyzing means (304) for analyzing the tree (120)of the segments (102-114) to control the first generating means.
 13. Animage processing apparatus (500) comprising: a motion estimator unit(300,301) for motion estimation of segments (102-114) of an image (100),comprising: a generator (310) for generating a set (118) of candidatemotion vectors of a particular segment (104); a computing means (306)for computing for each candidate motion vector a match penalty; and aselecting means (308) for selecting a particular motion vector (116)from the set (118) of candidate motion vectors on the basis of matchpenalties; and a motion compensated image processing unit (502),characterized in that the motion estimator unit (300,301) comprises: asegmentation means (302) for generating a tree (120) of the segments(102-114) of the image (100), designed to perform a hierarchicalsegmentation method; and an analyzing means (304) for analyzing the tree(120) of the segments (102-114) to control the generator (310).